Coverage measurement tools have been used to improve the quality of software and thereby prevent software failure. Conventional techniques for coverage measurement tools are described in, e.g., Japanese Patent Laid-open Nos. 5-334126 (Patent Document 1), 5-324402 (Patent Document 2), and 10-320190 (Patent Document 3).
A description will be given below of conventional coverage measurement techniques disclosed in these patent publications. It should be noted that even through some documents use the term “coverage analyzer” to refer to a tool for performing coverage measurement, this specification uses the term “coverage measurement tool” instead.
Coverage measurement tools have been used to check the quality of complete software. Coverage measurement tools have the so-called C0 coverage function and C1 coverage function. The C0 coverage function checks whether memory has been accessed by the program under test, while the C1 coverage function checks which branch of a conditional branch instruction of the program under test has been taken.
The C0 coverage is calculated based on the total number of instructions (or lines) to be executed in the program and the number of instructions (or lines) actually executed when the program was executed. The C1 coverage, on the other hand, is calculated based on the total number of branches to be taken in the program and the number of branches actually taken when the program was executed.
That is:C0 coverage (%)=(number of instructions actually executed)/(total number of instructions to be executed)×100, andC1 coverage (%)=(number of branches actually taken)/(total number of branches to be taken)×100.
Referring to FIG. 3, in C0 coverage measurement, if execution of a conditional branch instruction in the program under test causes the FALSE branch of the instruction to be taken, the coverage test fails to check the TRUE branch. This means that the test may be able to check only either the TRUE branch or FALSE branch. In the case of C1 coverage measurement, on the other hand, both branches can be comprehensively checked.
A description will be given below of an exemplary method for achieving coverage measurement, specifically C1 coverage measurement. When the program to be tested includes a conditional branch instruction (such as the instruction code 4 shown in FIG. 3), the compiler, assembler, etc. for the source program inserts a jump instruction (or possibly a software interrupt instruction) at a point immediately before the conditional branch instruction in order to jump to coverage measurement program code for performing C1 coverage measurement on the conditional branch instruction. (The instruction code 3 shown in FIG. 4 is the inserted jump instruction code.) When the program shown in FIG. 4 is executed, execution of the instruction code 3 causes processing to jump to the coverage measurement program code. The coverage measurement program checks the execution address of the instruction code 3 and the state of a CPU flag to determine whether the TRUE branch or FALSE branch of the instruction code 4 immediately succeeding the instruction code 3 will be taken when the instruction code 4 is executed. Then, processing returns to the program under test after storing the determination result.
The determination result, that is, data as to which branch of the instruction code 4 has been taken, is stored in a table as shown in FIG. 5. Specifically, if it is the TRUE branch, the coverage measurement program increments the value in the field 1710 in the table by one to indicate the number of times the TRUE branch of the instruction code 4 has been taken. If, on the other hand, the CPU flag indicates the FALSE branch, the coverage measurement program increments the value in the field 1711 in the table by one to indicate the number of times the FALSE branch has been taken. It should be noted that such a table is often generated by a compiler, assembler, etc. when it inserts the instruction code 3.
Upon completion of the execution, the tested program is processed based on the above stored data and the results are output. When each instruction code of the measurement target program was generated using high-level language code through a language processor such as a compiler, some coverage measurement tools also output information for associating each processing statement in the high-level language code with the above stored data as to which branch was taken and how many times in order to increase the test efficiency. After completion of the test, the inserted jump instruction (the instruction code 3) for coverage measurement is deleted to restore the original program code shown in FIG. 3. Then, the program is shipped as a software product.
The above method for coverage measurement has the problem of involving CPU overhead since it inserts a jump instruction at a point immediately before each target conditional branch instruction and executes it to determine the coverage. This overhead significantly reduces the execution speed, making it difficult to test a program which requires real-time processing. As a result, the test produces only reduced effect. This problem also arises with C0 coverage measurement.
Further, a conventional coverage measurement tool performs coverage measurement on the same branch repeatedly (that is, stores the number of times the branch has been taken) even when the branch is frequently taken. Repeated measurement on such a branch, that is, storing the number of times such a branch has been taken, is often unnecessary. The reason for this is that the calculation of C0 coverage or C1 coverage of a branch only requires information on whether the branch has ever been taken at least once (it does not matter how many times it has been taken). It is often unnecessary to check the same measurement point more than once. That is, preventing repeated measurement on branches that are frequently taken may lead to a reduction in the measurement time and an increase in the efficiency of the debugging process.
It should be noted that even though Patent Documents 1 to 3 each propose some technique for reducing the time required for coverage measurement, they do not satisfy the requirement of preventing repeated measurement on branches which are frequently taken.
It is, therefore, an object of the present invention to provide a method for testing a program, capable of preventing repeated measurement on branches that are frequently taken and thereby avoiding unnecessary overhead.